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L Real Party in Interest (37 § 1.1 92(c)(1)) 

The real party in interest in the present appeal is Microsoft Corporation, the assignee of 
the present application. 

IJ- Related Appeals and Interferences (37 C.F.R. § 1.192(c)(2)) 

Appellants, appellants' legal representatives, and/or the assignee of the present 
application are unaware of any appeals or interferences that will directly affect, or be directly 
affected by, or have a bearing on the Board's decision in the pending appeal. 

Ill- Status of Claims (37 CF-R. § 1-1 92(c)(3)) 

C1ainf)S 1-63 are currently pending in the present application. The rejection of claims 1-9, 
11,12, 14,16-22, 24, 33-40, 42, 44, 46-51. 53, 54, 56, and 58-63, and the objection to claims 10, 
13, 15, 23, 25-32, 41, 43, 45, 52, 55, and 57, are appealed. 

IV. Status ofAmendments (37 C.F.R- § ia92(c)(4)) 

No claim amendments have been made subsequent to the jfinal rejection of 
May 28, 2002. 

V. Summary of Invention (37 C^F.R, § 1.192(c)(5)) 

The claimed invention relates to generating visual representations of objects in a 
graphical user interface. More specifically, the invention relates to rendering a thumbnail such 
that depth of the thumbnail in a three-dimensional environment is a function of at least one 
parameter of an object associated yvith the thumbnail. Contrary to prior art techniques which 
render thumbnail as a function of the thumbnail itself, the subject invention provides for 
rendering of thumbnails as a function of an object (e.g,^ a dociiment, a spread sheet, a business 
contact, a drawing, a picture or image, a web page, a resource location or directory) represented 
by the thumbnail. See, e,g., page 23, line 23 to page 24, line 2, The claimed invention provides 
for a rich 3D experience since thumbnails are rendered as a function of parameter(s) of objects 
represented by the thumbnails that are meaningful to a user as compared to prior art systems that 
are in general solely concerned with display^space utilization. For example, via the claimed 
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invention thumbnails representing objects of greater importance {e.g.^ high priority documents, 
time sensitive matters. . .) can be rendered with a depth such thai ihey appear closer to a user than 
thumbnails representing objects of less relative importance. See, e.g., page 17. line 25 to page 
1 8, line 4, and page 23, lines 24-28. 

The claiined invention can map values of properties or parameters of the object 
information to a depth. The values of the properties or parameters of the object are unique to 
each object. See, e.g., page 42, lines 25-28. Using this depth information, the present invention 
renders the thumbnail of the information object in the simulated three-dimensional tank at a 
simulated depth. As values of properties or parameters of objects change, the simulated depth 
at which they are rendered can change. See, e.g., page 25, lines 4-6. Vahies of properties or 
parameters of the objects can be changed by the user or can be changed by via an algorithm in 
response to an indirect user action^ such as an absence of user action associated with a given 
object. See, e.g., page 45, lines 8-25. In this manner, the simulated depth of an object can 
change automatically. For example, a parameter associated with the object might be a due date. 
As the due date draws near^ so to does rendering of the object. This parameter of the object is 
not necessarily directly related to the object (e,g., as would be the physical size of the object, 
height times width.) 

VI. statement of the Issues (37 CF.R. § lJ92(c)(6)) 

Whether claims 1-9, 11,12, 14,16-22, 24, 33^, 42, 44, 46^51, 53, 54, 56, and 58-63 are 
patentable under 35 U,S.C. §103(a) over Robertson et aL (U.S. 6,166,738) and Joskowjcz et aL 
(U.S. 5,669,006). 

Applicants' representative acknowledges with appreciation the indication that claims 10, 
13, 15, 23, 25-32, 41, 43, 45, 52, 55 and 57 would be allowable if recast in independent forni to 
include all limitations of respective base claims and any intervening claims. The option to 
recast these claims in such form at a later date if necessary is respectfully reserved. 

VIL Grouping of Claims (37 § 1.192(c)(7)) 

For the purposes of this appeal only, the claims are grouped as follows: 
Claims 1-63 stand or fall together. 
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VIIL Argument (37 C.F.R. § 1.192(c)(8)) 



A. Reiectian of Claims 1-9. 1 K 12. 14,16-22. 24. 33^0, 42, 44, 46-51. S3. 54. 56. 
and 58-63 Updcr 35 Sl03fa^ 

Claims 1-9, 11,12, 14,16-22, 24, 33^0, 42, 44, 46-51, 53, 54, 56, and 58-63 stand 
rejected under 35 U,S.C. § 1 03(a) as being unpatentable over Robertson et al. and Joskowicz et 
at. A reversal of the rejection is respectfully requested for at least the following reasons- 
Neither Robertson et al. nor Joskowicz et aL^ alone or in combination, discloses every limitation 
of the claimed invention. The references, if combined as suggested by the Examiner, would not 
result in the invention as claimed. 

Applicable Lan^ 

To reject claims in an application under §103, an examiner must establish 
a prima facie case of obviousness. A prima facie case of obviousness is 
established by a showing of three basic criteria. First, there must be some 
suggestion or motivation, either in the references themselves or in the 
knowledge generally available to one of ordinary skill in the art, to 
modify the reference or to combine reference teachings. Second, there 
must be a reasonable expectation of success. Finally, the prior art 
reference (or references when combined) must teach or suggest all the 
claim limitations. See MPEP §706.020). The teaching or suggestion to 
make the claimed combination and the reasonable expectation of success 
must both be found in the prior art and not based on applicant's 
disclosure. See In re Vaeck 947 F.2d 488, 20 USFQ2d 1438 (Fed. Cir. 
1991). 

'Those charged with determining compliance with the Patent Act, 35 
U.S.C.S. §103, are required to place themselves in the minds of those of 
ordinary skill in the relevant art at the time the invention was made, to 
detennine whether that which is now plainly at hand would have been 
obvious at such earlier time. The invention must be viewed not with the 
blueprint drawn by the inventor, but in the state of the art that existed at 
the time. The invention must be evaluated not through the eyes of the 
inventor, who may have been of exceptional skill, but as by one of 
ordinary skill," Interconnect Planning Corporation v. Thomas E. Feil 
Robert O. Carpenter, V Band Systems, Inc., and Turret Equipment Corp,^ 
744 F.2d 1 132, 1 138; 227 USPQ 543. 
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The subject invenlion as recited in independent claim 1 recites "...determining a two- 
dimensional location and a depth of each of the thumbnails in Ihe ihree^imensional 
environment, wherein,ybr each of the thumbnails, the depth is a function of at least one 
parameter of the object associated with the thumbnail. . Independent claims 34, 61 and 62 
recite similar features as that noted above with respect to claim 1 . Neither Robertson, e( al. nor 
Joskowicz, et al, alone or in combination, teach or suggest such claimed feature of applicants* 
invention. Such feature of the claimed invention provides for mapping values of properties or 
parameters of object information lo a depth. Using this depth information, the present invention 
renders the thumbnail of the information object in a simulated three-dimensional tank at a 
simulated depth. Furthermore, as values of properties or parameters of the objects change, the 
simulated depth at which they are rendered may change. See page 17, line 25 to page 1 8, line 
4. Thus, the simulated depth at which a thumbnail is rendered can change without user 
intervention. For example, as a deadline approaches, the depth of an icon can be decreased by 
the subject invention. Such feature of the claimed invenlion provides for mapping values of 
properties or parameters of object information to a depth in a manner that automatically adjusts 
the depth of a thumbnail upon changes in parameter value. Accordingly, simulated depth of 
thumbnails respectively can be a function of a parameter (e.g., including but not limited to object 
priority, object age, object size (MB), preference for an object by a user relative to other objects, 
objeci status, object version, etc.) of respective objects. See, e-g,, page 63, line 27 to page 64, 
line 2. Thus, a thumbnail associated with a high-priority object can be rendered at less simulated 
depth {e.g.^ closer to viewer) than a thumbnail associated with a low-priority object. 

The cited references do not teach or suggest such feature. The cited references do not 
associate depth of a rendered thumbnail with at least one parameter of an object associated with 
the thumbnail. The Office Action dated May 28, 2002, at paragraph 9, page 1 1 , incorrectly 
advances that Joskowicz, et al. teaches such feature at col. 3, lines 55-col. 8, lines 51 » and Fig. 9, 
Rather. Joskowicz, et al simply teaches rendering of icons (including Z-axis based rendering) as 
a fiinction of the icons themselves (eg.^ size, area, predefined constraints relating to spacing of 
the icons), but not as function of at least one parameter of an object associated with the 
displayed thumbnail as defined in the claimed invention. As described in the specification of the 
present invention, parameters used to determine thumbnail depth can be fixed or can be 
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as^ciated with user input. The user input can be either direct (e.g., the user changes the 
parameter) or indirect {e.g.j the user takes some action that resuU$ in a parameter change, such as 
clicking on thumbnail(s)). See, for example, page 44, line 1 1 to page 45, line 25. 

The Advisory Action dated November 27, 2002 relies on Joskowicz e( al. alone to 
disclose the element of using the *'sjze^' as a parameter of an object to directly proportion the 
depth of the object, wherein a user may later directly alter the depth of the object. However, 
Joskowicz et aL does not teach or suggesT rendering an icon as a function of at least one 
parameter of an object associated with the displayed icon as "parameter"* as defined in the 
present, apphcaiion, nor does it teach rendering the icon in a manner that permits automatic depth 
adjustment in response to a change in the parameter. Rather, Joskowicz et al. merely suggests 
"z-ordering" thumbnails by surface area so that larger thumbnails do not obscure smaller 
thumbnails. This is to be distinguished firom using file size {e.g. megabyte value, not physical 
size) as a parameter for determining depth placement of a thumbnail. That is, Joskowicz et aL 
teaches rendering an icon as a function of a parameter associated directly with the icon^ i.e. the 
depth of the icon is directly related to the physical size of the icon (height times width). The 
present invention renders an icon as a function of a parameter of an object associated with the 
icon^ wherein the icon is rendered as a function of an object {e.g.^ a deadline associated with the 
icon) represented by the icon rather than the icon itself See, e.g.^ claim 1. See also, e.g., page 
65, lines 11-28. 

The Examiner's Answer dated August 26, 2003, contends that 'In contrast to Appellants* 
argument, Robertson and Joskowicz teach the depth as a function of a parameter (such as priority 
or arrangemejit of episodes) of the object associated with the thumbnail." (Page 18, lines 1 8-20.) 
Only once is the word "priority" mentioned in the Joskowicz patent, which mention occurs at 
Column 4, line 22, The "priority** of which Joskowicz et al speaks is the priority of the 
constraints (f.e., rules) by which icon rendering is governed. For example, according to 
Joskowicz etal^ the requirement that "a button should always be fiilly visible*' supercedes the 
requirement that "a portion of text may not be hidden bdiind another episode." If these two 
physical features of the thumbnails themselves are in conflict, Joskowicz et al requires that the 
button be made visible at the cost of hiding text behind another episode. Hiis **prioril/* is not a 
function of a parameter of an object assodated with the thumbnail, but rather is merely a 



6 



PAGE 8125 * RCVD AT 1/2312004 6:35:26 PM [Eastern Standard Time] * SVR:USPT^^^^^ 



01/23/2004 Id: 32 FAI 216 896 $731 

09/335,640 



AMIN, & TUROCY LLP, 



[g)009 



MS J 16627.1 



function of a rule set for detennining. what portion of a Ihumbnail will be made visible based on 
the physical properties of the thumbnail itself. 

Furthermore, Robertson et aL and Joskowicz et al. both are concerned with rendering of 
thumbnails/icons so. as to optimize utilization of valuable screen real estate via three-dimensional 
rendering of the thumbnails. However, neither reference teaches or suggests that the placement 
of the thumbnails in three-dimensional space is also a function of at least one parameter of art 
object associated with the rendered thumbnail. The subject claimed invention provides for three- 
dimensional rendering of thumbnails so as to provide for organi2ing and rendering the 
thumbnails as a function of relevance (as defined, for example, by the number of clicks since last 
use by the user) of the objects associated with the thumbnails. See, e.g., page 45. lines 8-25. 
Such aspect of the claimed invention provides for optimizing rendering of the thumbnails not 
only with respect to utilization of valuable screen real estate but also with respect to relevance of 
the respective objects (associated with the thumbnails) to a user as defined by at least one 
parameter ofthc respective objects. See, e.g., page 45, lines 8-25. Furthermore, **[a]s shov^^ in 
Figure 3» an object record 304 may include an object idetitifier field 306 which includes a value 
used to distinguish the object from all other objectsr (Page 42, lines 22-25.) Neither of the 
cited references teaches or suggests this aspect of the present invention. Thus, the claimed 
invention provides for a richer and more refined technique for rendering thumbnails that is not 
disclosed, suggested or even contemplated by the cited references. 

Additionally, Joskowicz et ah does not teach a "depth parameter of the object thumbnails 
on the three dimensional surface" or '^three-dimensional features of an object, such as depth 
infomnation," "Depth,** as used in Joskowicz et al^ is a vector of elements that are ordered from 
back to front without three-dimensional perspective (Column 5, lines 4-5). The depth of 
Joskowicz et al. does not have a perspective. Depth is only a Z-ordering (Column 3, lines 37- 
38), and the Z-ordering is only an order of objects from back to front (Column 4, lines 3-10). 
The constraint Z(I) g 2(1+1) is a constraint on the order in which elements are inserted into a Z- 
list. No function (linear or polynomial) for calculating a three dimensional depth is taught or 
suggested in Joskowicz et aL Element Ei « Eii j is merely a statement of relative position, but 
has no affect on how either element is rendered. Jn particular, the elements are not scaled to 
create the illusion of depth. The 2-order is merely for calculating which elements occlude 



7 



PAGE 9125 * RCVD AT 112312004 6:35:26 PM [Eastern St^^^ 



01/23/2004 ia:33 FAX 216 696 8731 AMIN, & TUROCY LLP. Si 010 



09/335,640 MS 11 6627 J 



others- Note that the bounds on area are completely independent of depth (Column 5, line 65). 
Also, the ^yo^d ^'perspective" does not appear in Joskowicz etai and the Z dimension, which is 
an order, is different from the X and Y dimensions, which are true coordinate systems (Column 
3, lines 3 1-35). 

"A prior art reference must be considered in its entirety, i.c.^ as a whole, 
including portions that would lead away from the claimed inveniion." 
W.L. Gore <& Associates, Inc. v. Garlock, Inc., 721 F.2d J 540, 220 
USPQ 303 (Fed.-Cir. 1 983), cert, denied, 469 U.S. 85 1 (1 984) 

In fact, Joskowicz et aL teaches away from the subject invention and from Robertson. In 
particular, Joskowicz et aL teaches of putting elements with larger areas at the back of the z- 
order (with no scaling for perspective) and the s maller elements at the fronts which teaches away 
from a perspective view in which farther away elements appear smaller, as set forth in the 
subject application (Column 7, lines 27-30)_ A reference that teaches away from another 
reference may not be combined there with to form the basis of a 35 U,S.C- § 103 rejection. {See, 
e.g,. In Re Grasselli, 713 R2d 73J, 218 USPQ 769, 779, Fed. Cir. 1983; "It is improper to 
combine references where the references teach away from their combination.") 

As per claim 2 and 3, the depth of Joskowicz et aL is never calculated with a linear or 
polynomial function. Only the constraints on the area of the different objects in the cliques are 
disclosed in Column. 4, line 52-Column 6^ line 62. Also, Joskowicz et aL does not disclose 
three-dimensional depth, as discussed above. 

In view of at least the above, it is respectfully submitted that the subject invention as 
recited in independent claims 1, 34, 61 and 62 (and claims 2, 3, 5-9, 11, 12, 14, 16-22, 24, 33, 
35-40, 42, 44, 46-51, 53, 54, 56, and 58-60 and 63 which respectively depend diere from) is not 
obvious over the combination of Robertson^ et aL and Joskowicz, et aL Specifically, claims 5, 6, 
7, 8, 9^ 11, 12, 14, 16, 27, 18, 19, 20, 21, 22, 24, 33, 35-40, 42, 44, and 46 depend upon a 
rejected base claim that is believed to be allowable. Therefore, allowance of these claims is 
respectfiilly requested. Claims 47,48,49,50,51,53,54,56,58,59, and 60 reference a claim that is 
believed to be allowable, and therefore allowance of these claims is also respectfully requested. 
Claims 61 and 63, which correspond generally to claim 1 , are believed to be allowable for the 
above reasons. Claim 62 corresponds to claim 34 and is believed to be allowable for the above 
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reasons- 



B. Rejection of Claim 4 Under 35 U>S>C- gl03(a) 

Claim 4 stands rejected uj}der 35 U.S.C. 103(a) as being unpatentable over Robertson, et 
al. and JosVowicz, el al and Baldwin (U.S. 5,701,444), Withdrawal of this rejection is 
respectfully requested for at least the following reasons. 

Claim 4 depends from independent claim 1, and Baldwin fails to make up for the 
aforementioned deficiencies of Robertson, ei al and Joskowicz, ex ai with respect to claim 1 . 
Thus, this rejection should be withdrawn. 

IX. Conclusion 

It is readily apparent from at least the above comments that neither Robertson e/ al. nor 
Joskowicz et ai, alone or in combination, discloses applicants' invention as recited in 
independent claims 1, 34, 61, and 62. Claims 2-33, 35-60, and 63 depend directly or indirectly 
from independents claims 1 , 34, and 61 . Accordingly, a reversal of the rejection of claims 1-9, 
] M 2, 14,1 6-22, 24, 33-40, 42, 44, 46^5 1 , 53, 54, 56, and 58-63, and a removal of the objection 
to claimslO, 13, 15, 23, 25-32, 41, 43, 45, 52, 55, and 57, is respectfiilly requested. 
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Respectfully submitted, 
AMIN&TUROCY, LLP 




Himanshu S. Amin 
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X. Appeudut of Claims (37 C.F,R- § 1.1 92(c)(9)) 

1 , A man-inachine interface method for permitring a user to act on thumbnails, each 
ihumbnail rq)resenting an assodated object containing information, for use with a machine 
having a video display device and a user input device^ the man-machine interface method 
comprising: 

a) geiiCTating a three-dimensional environment, having a depth, to be rendered on the 
video display device; 

b) determining a two-dimensional location and a depth of each of Ihe thumbnails in the 
three-dimensional environment, wherein, for each of the thumbnails, the depth is a function of at 
least one parameter of the object associated with the thumbnail; and 

c) generating the thumbnails within the three-dimensional environment, at the 
determined two-dimensional locations and depths, to be rendered on the video display device. 

2, The man-machine interface method of claim 1 wherein, for each of the thumbnails 
the depth is a linear function of at least one parameter of the object associated with the 
thumbnail. 

3. The man-machine interface method of claim 1 wherein, for each of the thumbnails, the 
depth is a polynomial function of at least one parameter of the object associated with the 
thumbnail. 

4. The man-machine interface method of claim 1 wherein, for each of the thumbnails, the 
depth is an exponential function of at least one parameter of the object associated with the 
thumbnail. 

5. The method of claim I wherein the at least one parameter includes at least one parameter 
selected from a group of parameters consisting of (a) click history, (b) age, (c) time since last 
use, (d) size, (e) file type, (f) associated application, (g) classification, and (h) author. 

6, The man-machine interface method of claim 1 further comprising: 
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d) accepting inputs from the user input device; 

e) determining a two-dimensional cursor location based on the accepted inputs; and 

f) generating a cursor at the determined two^imensional cursor location, to be rendered 
on the video display device, 

7. Tlie man-machine interface method of claim 6 further comprising: 

g) if the two-dimensional location of the cursor is located on or over one of the 
thumbnails, defining a state of that thumbnail as active. 

8. The man-machine interface method of claim 7 further comprising: 

h) generating a pop-up information bar located over the active thumbnail, to be rendered 
on the video display device. 

9. The man-machine interface method of claim 7 further comprising: 

h) if the user input provides a selection input and if an active or floated thumbnail exists, 
then generating a higher resolution visual representation of the object represented by and 
assodated with the active or floated thumbnail, at a preferred viewing location at a foreground of 
the three dimensional environment, to be rendered on the video display device. 

JO. The man-machine interface method of claim 7 further comprising: 

h) if the user input provides a float input and if an active thumbnail exists, then setting 
the deplli of the active thumbnail lo a predetemfiined value and defining a state of the active 
thumbnail as floated. 

1 1 . The man-machine interface method of claim 9 wherein ihe act of generating the higher 
resolution visual representation of the object represented by and associated with the active 
thumbnail includes: 

- generating an animation which moves the higher resolution visual representation 
of the object represented by and associated with the active thumbnail from the 
location of the active thumbnail to the preferred viewing location at the 
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foreground of the three dmiensional eDvironment, to be rendered on the video 
display device. 

4 2. The man-machine inCerface method of claim 1 1 further comprising: 

i) if the user input provides a deselection input and if a selected thumbnail exists, then 
generating a video output for moving the high resolution visual representation of the object 
represented by and associated with the active thumbnail to the two-dimensional location of the 
selected thumbnail, to be rendered on the video display device. 

1 3. The man-machine interface method of claim 9 further comprising: 

i) if the user input provides a sink input and if a floated thumbnail exists, then setting the 
depth of the floated thumbnail to a previous value and defining a state of the floated thumbnail as 
active. 

14. The man-machine interface method of claim 7 further comprising: 

h) if the user input provides a selection input and if an active thumbnail exists, then 

i) invoking an application related to the object represented by and associated with 
the active thumbnail, 

ii) loading the object represented by and associated with the active thumbnail into 
the application, and 

iii) generating a video output of the application with the loaded object represented 
by and associated with the active thumbnail at a preferred viewing location, to be 
rendered on the video display device, 

1 5. The man-machine interface method of claim 9 further comprising: 

h) if the user input provides a selection input and if a floated thumbnail exists, then 

i) invoking an application related to the object represented by and associated with 
the floated thumbnail, 

ii) loading the object riepresented by and associated with the floated thumbnail 
into the application, and 
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iii) generating a video output of the application with the loaded object represented 
by and assodated with the floated thumbnail at a preferred viewing location, to be 
rendered on the video display device. 

1 6. The man-machine interface method of claim 7 further comprising: 

h) if the user input provides a move input and if an active or floated thumbnail exists, 

then 

i) updating the two-dimensional location of the active or floated thumbnail based 
on the move input, 

17. The man-machine interface method of claim 16 wherein the moveinput is a left button 
mouse drag. 

1 8. The man-machine interface method of claim 1 wherein the three-dimensional 
environment de^ncs a foreground and a background, and 

wherein the act of generating thumbnails, within the three-dimensional environment, at 
the determined two-dimensional locations and depths, to be rendered on the video display 
device, includes: 

i) using perspective views so that any thumbnails in the foreground defined by 
the three-dimensional environment appear larger than any thumbnails in the 
background defined by the three-dimensional surface. 

1 9- The man-machine interface method of claim 18 wherein a thumbnail partially occludes 
any thumbnails behind it, based on a viewing poinL 

20. The man-machine interface method of claim 1 further comprising: 

d) accepting inputs from the user input device; 

e) determining a viewing point two-dimensional location, depth and direction based on 
the accepted inputs; and 
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f) generating only that portion of the three-dimensional environment and only those 
thumbnails that are in front of the virtual viewing point determined in act (e), to be rendered on 
thcr video display device. 

21 . The man-machine interface method of claim 1 wherein the thumbnails are low resolution 
bitmaps. 

22. The man-machine interface method of claim 2 1 wherein the low resolution bit maps are 
64 pixels by 64 pixels and have 24 bit color. 

23. The method of claim 20 wherein if the depth of the viewing point is below a 
predetermined depth, fiirther performing a step of: 

g) gradually decreasing the depth of the viewing point to float the viewing point while no 
user inputs are received. 

24. The method of claim 1 further comprising, for each of the thumbnails, determining a 
shade to be applied to the thumbnail based on its depth. 

25- The method of claim 24 wherein the shade to be applied to the thumbnail darkens as the 
depth increases. 

26. The method of claim 24 wherein the shade to be applied to the thumbnail darkens as a 
distance between the depth of the thumbnail and a viewing point increases. 

27. The method of claim 1 fiiaher comprising, for each of the thumbnails, determining a fade 
to be applied to the thiimbnail based on its depth. 

28. The method of claim 27 wherein the fade to be applied to the thumbnail increases as the 
depth increases. 
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29. The method of claim 27 wh^ein the fade to be applied to the thumbnail mcreases as a 
distance between the depth of the thumbnail and a viewing point increases, 

30. The method of claim 1 further comprising, for each of ihe thumbnails, dctennining a tint 
to be applied lo the ihumbnail based on its depth. 

3 1 . The method of claim 30 wherein the tint to be applied to the ihtimbnail increases as the 
depth increases. 

32. The method of claim 30 wherein the tint to be applied to the thumbnail increases as a 
distance between the depth of the thumbnail and a viewing point increases. 

33* The method of claim 1 wherein the three dimensional environment includes a floor, the 
method further comprising a step of generating a shadow, for each of the thumbnails, on the 
floor. 

34, A system which permits a user to interact with thumbnails, each thumbnail representing 
an associated object containing information, the system comprising: 

a) an input facility for accepting user inputs; 

b) a storage facility containing 

i) a two-dimensional location, a depth and state information for each of the 
thumbnails, 

ii) a two-dimensional cursor location, and 

iii) a three-dimensional environment having a simulated depth; 

c) a processing unit which 

i) accepts user inputs from the input facility, 

ii) updates (a) the two-dimensional location, and state information for each of the 
thumbnails contained in the storage facility, and (b) the two-dimensional cursor 
location contained in the storage fadlity, based on the accepted user inputs, 
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iii) updates depth information for each of the thumbnails contained in the storage 
facility based on at least one parameter of the object associated with the 
thumbnail, and 

iv) generates video oxjtputs based on 

A) the two-dimensional location^ depth and state information for each of 
the thumbnails, 

B) the two-<limensional cursor location, and 

C) the three-dimensional environment, 
contained in the storage facility; and 

d) a video display unit for rendering the video outputs generated by the processing unit. 

35. The system of claim 34 wherein the state infomiation for each of the thumbnails 
contained in the storage facility includes an indication of whether or not the thumbnail is active, 
and 

wherein ihe processing unit determines that a thumbnail is active if a cursor is located on 
or over a thumbnail based on the two-dimensional location of the cursor and the two dimensional 
location of the thumbnail. 

36, The system of claim 34 wherein the storage facility further contains descriptive textual 
information for each of the thumbnails, and 

wherein, if a thumbnail is active, 

i) the processing unit generates a pop-up bar, based on descriptive textual 
infoimation, for the active thumbnail, and 

ii) the video display unit renders the pop-up bar over the rendered thumbnail. 
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37. Ilie system of claim 35 wherein, if a thumbnail is active or floated and the input facility 
accepts a selection input, then 

i) ihe processing unit vpdates the state of the thumbnail, 

ii) the processing unit gets a second, higher resolution, visual representation of 
the object represented by and associated with the thumbnail^ 

iii) the processing unit generates a video output based on the higher resolution, 
visual representation of the object represented by and associated with the 
thumbnail at a preferred viewing location, and 

iv) the video display device renders the video output generated by the processing 
unit 

38. The system of claim 37 further comprising an audio output device, 
wherein the storage facility further contains a first audio cue, and 

wherein, when an object is selected, the processing unit provides the jfirst audio cue to the 
audio output device. 

39. The system of claim 37 wherein each thumbnail is a 64 pixel by 64 pixel bit map having 
24 bit color and wherein each higher resolution, visual representation of the objects is a 512 pixel 
by 51 2 pixel bit map having 24 bit color. 

40. The system of claim 37 wherein the processing unit further effects a video output based 
on an animation of the higher resolution, visual representation of the object represented by and 
associated with the thumbnail, moving from the location of the thumbnail to a location at the 
foreground of the three-dimensional environment. 

41 . The system of claim 35 wherein if the input fadlity provides a float input and an active 
thumbnail exists, then the processing unit will set the depth of the active thumbnail to a 
predetermined value and will define the stale of the active thumbnail as floated. 
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42. The system of claim 35 wherein, if a thumbnail is active and the input facility accepts a 
selection mpm, then 

i) the proceiising unit updates the state of the thumbnail to selected, 

ii) the processing unit opens an application with which the object, assodated with 
and represented by the selected thumbnail, is associated, 

iii) the processing unit loads the object into the application, 

iv) the processing unit generates a video output based on the object loaded onto 
the opened application and a preferred viewing location, and 

v) the video display device renders the video output generated by the processing 
unit, 

43. The system of claim 41 wherein if the input facility provides a sink input and if a floated 
thumbnail exists, then the processing unit will set the depth of the floated thumbnail to a 
previous value and will define a state of the floated thumbnail as active. 

44. The system of claim 37 wherein, if a thumbnail is active or floated and the input facility 
accepts a move input, then 

i) the processing unit updates the slate and location of the thumbnail, 

ii) the processing unit generates a video output based on the updated location of 
the thumbnail, and 

iii) the video display device renders the video output generated by the processing 
unit. 

45. The system of claim 35 wherein if a thumbnail is floated and the input facility accepts a 
selection input, then 

i) the processing unit updates the state of the thumbnail to selected, 

ii) the processing unit opens an application with which the object, associated with 
and represented by the selected thumbnail, is associated, 

iii) the processing unit loads the object into the application. 
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iv) the processing unit generates a video output based on the object loaded onto 
the opened application and a preferred viewing location, and 

v) the video display device renders the video output generated by the processing 
unit. 

46. The system of claim 34 wherein the storage facility further contains virtual viewing point 
location information, 

wherein the input facility includes a mouse, and 
wherein the processing unit 

d) accepts inputs from the user input device; 

e) determines a viewing point location and direction based on the accepted inputs; and 

f) generates only that portion of the three-dimensional environment and only those 
thumbnails that are in front of the virtual viewing point determined in step (e), to be rendered on 
the video display device. 

47. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, perfonns the method of claim 1 . 

48. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, perfonns the method of claim 6. 

49. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of claim 7. 

50- A machine readable medium contaiiiing data and machine executable instructions which, 
when executed by a machine, performs the method of claim 8, 

51. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine^ performs the method of claim 9. 
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52. A machine readable medium containing data and machine executable instructions which, 
when executed by a niachine^ performs the method of claim 1 0. 

53. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of cl aim 1 1 . 

54. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of claim 1 2. 

55. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs themethod of claim 13. 

56. A machine readable medium containing data and machine executable instructions which, ^ 
when executed by a machine, performs the method of claim 14_ 

57- A machine readable medium containing data and machine executable instructions which, 
when executed by a machine^ performs the method of claim J 5. 

58. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of cjaim 1 8- 

59. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of claim 1 9. 

60. A machine readable medium containing data and machine executable instructions which, 
when executed by a machine, performs the method of claim 20- 

61. A man-machine interface method for permitting a user to act on thumbnails, each 
thumbnail representing an associated object containing information, for use with a machine 
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having a video display device and a user jnpiit device, the man-machine interface method 
comprising: 

a) generating a ihree-dimensional environment, having a depth, to be rendered on the 
video display device; 

b) determining a two-dimensional location and a depth of each of the thumbnails in the 
three-dimensional environment, wherein, for each of the thumbnails^ the depth is a fiinction of at 
least one property of the object associated with the thumbnail; and 

c) generating the thumbnails within the three-dimensional environment, at the 
determined two-dimensional locations and depths, to be rendered on the video display device. 

62. A system which permits a user to interact with thumbnails, each thumbnail representing 
an associated object containing information^ the system comprising: 

a) an input facility for accepting user inputs; 

b) a storage facility containing 

i) a two-dimensional location, a depth and state information for each of the 
thumbnails; 

ii) a two-dimensional cursor location, and 

iii) a three-dimensional environment having a simulated depth; 

c) a processing unit which 

i) accepts user inputs from the input facility, 

ii) updates (a) the two-dimensional location, and state information for each of the 
thumbnails contained in the storage facility,- and (b) the two-dimensional cursor 
location contained in the storage facility, based on the accepted user inputs, 

iii) updates depth information for each of the thumbnails contained in the storage 
facility based on at least one property of the object associated with the thumbnail, 
and 

iv) generates video outputs based on 

A) the t\yo*dimensional location, depth and state information for each of 
the thumbnails, 

B) the two-dimensional cursor location, and 
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C) the three-dimensional environment, 
contained in the storage facility; and 
d) a video display unit for rendering the video outputs generated by the processing unit. 

63- A machine readable medium containing data and machine executable inslruciions which, 
when executed by a machine, performs the method of claim 61 . 
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